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EXECUTIVE  SUMMARY 


1.  State  of  the  Art.  In  large  military  command  and  control  configurations, 
European  decision-aiding  technology  is  appreciably  behind  tne  American  state  of  the 
art — primarily  because  of  the  scale  involved.  US  forces  and  technological  effort  are 
often  an  order  of  magnitude  greater  than  similar  resources  in  European  countries. 

2.  Medium-Size  Systems.  Europeans  are  doing  important  research  in  medium-size 
decision  aiding  systems.  Much  of  the  work  falls  into  three  classes: 

•  Unburdening  systems.  Aids  in  this  category  perform  routine  processing,  thus 

freeing  the  human  operator  for  high-level  control  and  decision  tasks.  There  are 
specific  aiding  systems  for  aircraft  and  submarine  control,  handicapped  people, 
power  plants,  continuous  process  manufacturing,  in-flight  monitoring,  and  many 
other  applications.  European  theoretical  work  in  estimating  human  mental  capa¬ 
city  is  at  the  edge  of  the  art. 

•  Diagnostic  systems.  Expert  systems  and  other  computerized  diagnostic  aids  attempt  to 

condense  in  a  compact,  portable  package  an  individual's  or  a  technology's 
specialized  information.  European  prototypes  include  the  Pourbaix-Michie 
metallurgical  expert  system,  now  under  development  at  Oxford,  and  the 
anesthesiology  support  system  at  Aachen,  West  Germany.  One  clear  trend  is 
discernible  in  many  diagnostic  aids:  human  "mental  models"  of  basic  physical 
processes  are  included  in  the  systems.  ONR  Code  440  contractors  probably  should 
keep  abreast  of  this  work,  and  explicit  ONR  technical  report  exchanges  should  be 
initiated  with  major  European  laboratories  working  in  the  area. 

•  Structuring  systems.  For  many  complex  problems,  the  most  effective  system  does  not 

attempt  to  compute  a  "best  answer"  or  tell  you  what  to  do  next;  rather,  it 
organizes,  displays,  and  tests  the  inputs  so  that  the  structure  of  a  problem 
becomes  more  evident  to  the  decision  maker.  Such  aiding  packages  are  general  and 
content-free;  they  elicit  systematic  probability  and  utility  judgments  from 
people.  One  working  European  system  is  MAUD  (London  School  of  Economics),  and 
there  are  several  others.  ONR  Codes  411  and  442  should  consider  a  project  to 
compare  systematically  European  structuring  packages  with  American  ones;  such  a 
review  probably  would  require  two  specialists,  a  software-oriented  decision 
analyst  and  a  psychologist  familiar  with  utility  models. 

3.  Graphics.  Interactive  graphics  capabilities  are  gradually  being  added  to 
European  decision-aiding  packages  as  computational  power  increases.  Within  a  short 
time,  say  3  to  5  years,  stand-alone  computerized  aids  will  have  significant  drawing, 
display,  and  perspective  transformation  capabilities.  ONR  Code  442  contractors — who  are 
already  at  the  forefront  in  antiaircraft,  antisubmarine  warfare,  and  electronic  warfare 
aiding  research — could  maintain  their  lead  in  this  field  by  pursuing  a  strong  program  on 
the  use  of  interactive  graphics  for  decision  aiding.  Advanced  graphics  packages  are 
already  being  marketed  by  Japanese  firms. 

4.  Criterion  and  Implementation  Problems.  Some  decision  aids  are  used  routinely 
and  effectively;  the  diagnostic  tests  run  in  large  computer  centers  are  one  example.  In 
general,  however,  neither  the  effectiveness  nor  likelihood  of  acceptability  of  a  given 
aiding  system  can  be  predicted.  Forums  such  as  the  annual  International  Symposium  on 
Applied  Military  Psychology  and  certain  NATO  and  Advisory  Group  for  Aerospace  Research 
and  Development  panels  are  examining  the  problems;  the  whole  field  would  benefit  from  an 
in-depth  survey  of  what  is  known  and  what  is  conjectured.  Some  European  defense  minis¬ 
tries  and  computer  societies  have  appointed  special  panels  on  these  topics.  A  timely 
ONR  review  of  implementation  sponsored  by  several  Codes  would  be  valuable. 


5.  General  Versus  Specific  Aids.  Most  European  aids  are  tailored  for  decisions 
that  must  be  made  in  a  particular  setting.  Z-.  is  often  envisioned  th.it  a  specialist 
equipped  with  a  portable  aiding  system  can  go  to  the  scene,  perform,  and  leave.  An 
alternative  concept  is  to  have  a  general  aiding  system  with  several  different  unbur¬ 
dening  and  diagnostic  schemes,  which  can  be  called  up  and  used  by  many  kinds  of  special¬ 
ists  who  insert  their  data  bases  into  it.  The  latter  option  becomes  more  feasible  as 
small  computers  increase  in  power  and  as  it  becomes  clear  that  rather  few  models  under¬ 
lie  much  of  the  aiding  logic.  ONR  should  consider  research  to  investigate  the  utility 
of  general  versus  specific  aiding,  because  the  choices  will  have  profound  implications 
for  future  personnel  and  training  policies  of  the  Navy. 

6.  Knowledge  Representation.  European  decision  aids  have  many  frameworks  for 
expressing  "knowledge"  about  a  domain.  Every  major  country  is  doing  studies  to  estab¬ 
lish  knowledge  models  that  will  be  "user  friendly."  However,  there  seem  to  be  few 
similarities  between  (and  little  use  of)  models  in  the  knowledge  representation  field. 
ONR  should  consider  sponsoring  a  review,  or  perhaps  an  international  meeting,  to  take 
stock  of  this  active  area. 
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DECISION  AIDING  IN  EUROPE:  ASSESSMENT 
REPORT 

A  decision  aid  can  be  anything  from  a 
hand-held  nomograph  or  gadget  to  a 
full-scale  command  and  control  system. 
Some  of  the  most  interesting  European  work 
is  being  done  with  relatively  small  compu¬ 
terized  aiding  systems.  Such  systems  often 
are  designed  for  a  military,  technical,  or 
industrial  situation  in  which  a  human 
decision-maker  is  clearly  designated  but 
the  best  decision  is  not  obvious.  Some¬ 
times  the  aid  includes  a  dedicated  computer 
and  special  software,  and  often  there  is 
some  abstract  model  of  the  decision  situa¬ 
tion  that  serves  as  the  logical  basis  of 
the  aid. 

Technological  interest  in  such  aids 
stems  from  the  hope  that  decisions  can  be 
made  more  effectively,  and  from  the  pos¬ 
sibility  that  proven  techniques  can  be 
transferred  rather  directly  from  estab¬ 
lished  disciplines  such  as  applied  statis¬ 
tics,  control  theory,  and  decision  analy¬ 
sis.  There  are  also  many  basic  scientific 
questions  involved  in  the  representation 
and  processing  of  knowledge,  the  interac¬ 
tive  dialogue  between  man  and  aid,  the 
prediction  of  aiding  effects,  and  the 
parameterization  of  subjective  values. 

Why  is  aiding  necessary?  There  can  be 
many  reasons,  but  at  least  three  stand  out: 

1.  The  human  operator  is  ''overloaded" 
and  cannot  process  efficiently  all  the 
information  coming  in.  Typical  situations 
include  monitoring  many  aircraft  tracks  or 
communication  channels  at  once,  or  select¬ 
ing  weapon  allocations  during  a  fast-moving 
battle.  An  aid  might  do  much  of  the 
routine  processing  semiautomatically,  and 
thus  unburden  the  operator  so  that  he  could 
handle  better  the  parts  of  the  task  that 
must  be  done  by  humans.  For  convenience, 
we  call  these  "unburdening"  systems. 

2.  The  operator  does  not  know  enough, 
or  cannot  remember  enough,  about  the 
underlying  process.  An  example  is  the 
diagnosis  of  failure  in  a  complex  physical 
or  biological  system.  A  competent  techni¬ 
cian  might  know  a  lot  about  the  action  of 
electronic  components  and  circuits,  but  a 
guided  missile  or  radar  is  so  complex  that 
no  one  could  remember  everything  that  might 
need  to  be  checked.  An  aid  could  help  by 


separating  the  system  into  mentally 
manageable  units;  suggesting  tests  to  be 
done,  given  the  information  obtained  so 
far;  and  keeping  track  of  the  whole  diag¬ 
nostic  effort.  Such  an  aid  also  might  be 
of  use  in  scheduling  and  resource  allo¬ 
cation,  for  which  long  lists  of  facili¬ 
ties  and  requirements  need  to  be  merged, 
ordered,  and  updated.  Though  the  label 
is  not  perfectly  descriptive,  we  call 
such  aids  "diagnostic." 

3.  The  decision-maker  does  not 
fully  understand  the  problem,  and  its 
probable  consequences.  One  example  is  a 
messy  multi-criterion  problem  with  many 
payoffs,  options,  and  uncertainties.  The 
intrinsic  complexity  and  vagueness  might 
cause  a  person  to  vacillate  about  what 
the  problem  really  is,  and  about  what 
weights  and  numbers  should  be  assigned  to 
likelihoods  and  events.  An  aid  providing 
a  general  framework  for  managing  such 
problems  would  elicit  and  combine  judg¬ 
ments  from  the  decision-maker;  the 
information  would  be  organized  and 
displayed  so  that  it  could  be  readily 
understood.  The  aid  might  help  the 
decision-maker  see  implications  of  the 
judgments  that  has  been  made.  And  the 
user  probably  would  be  able  to  adjust  the 
displayed  structure  as  he  gains  experi¬ 
ence.  The  system  might  be  used,  for 
example,  to  analyze  the  conflicts  likely 
to  occur  when  various  policies  are 
decided.  Such  aids  are  called  "structur¬ 
ing"  systems. 

The  three  kinds  of  aiding  require¬ 
ments  frequently  overlap;  in  a  specific 
aiding  project,  all  three  may  have  to  be 
faced.  But  they  do  furnish  an  informal 
scheme  for  reviewing  unclassified  Euro¬ 
pean  research  on  decision  aids.  There 
seems  to  be  no  single  compilation  of 
European  aiding  efforts,  though  military 
and  other  governmental  agencies  have 
occasionally  assembled  references  to 
specific  aids — say  for  air  control  or 
group  decision  making.  This  report 
describes  some  European  projects  and 
ideas  that  may  interest  American  re¬ 
searchers  now  working  in  the  area. 
Little  of  the  work  is  in  the  regular 
journal  literature,  as  it  tends  to  appear 
in  technical  reports  and  in  preliminary 
papers  at  meetings.  Just  enough  documen- 
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tation  is  given  here  to  provide  the 
interested  specialist  leads  for  further 
inquiry.  Usually,  detailed  information 
about  investigators  and  projects  can  be 
obtained  from  the  author  at  ONR-London. 

Unburdening  Systems 

European  work  in  military  unburdening 
configurations  does  not  appear  to  be  ahead 
of  American  research  in  any  basic  sense. 
Central  military  command-control  centers 
are  now  operating  in  all  the  major  coun¬ 
tries  and  at  Supreme  Headquarters  Allied 
Powers,  Europe  (SHAPE)  in  Belgium.  The 
writer  has  not  seen  them,  but  unclassified 
information  indicates  that  the  centers  are 
not  innovative  by  American  standards, 
except  for  an  occasional  display  or  key¬ 
board  layout.  Geographic  plots,  force 
status  indicators,  automatic  initialization 
of  tracks,  and  operational  command  arrange¬ 
ments  reportedly  resemble  those  in  American 
command,  control,  and  communication  cen¬ 
ters — though  for  a  single  European  nation 
the  features  can  be  much  simpler  because  of 
the  smaller  scale.  A  country  like  Holland 
or  Norway,  for  example,  has  rather  few 
ships  and  submarines.  And  even  though  the 
UK's  Royal  Navy  mounted  a  successful 
campaign  in  the  Falklands,  it  has  only 
about  one  tenth  of  the  personnel  strength 
of  the  US  Navy  and  operates  only  one  or  two 
aircraft  carriers.  Smaller  forces  do 
permit  a  leaner  command  structure;  de¬ 
cisions  can  often  be  made  very  quickly 
because  there  are  few  layers  of  command. 

The  Moscow  air  authority  is  now 
evaluating  a  new  air  control  system  that 
was  built  in  Sweden  according  to  Russian 
task  analyses  and  specifications.  Last 
summer,  an  American  team  was  invited  to 
Russia  to  see  the  system  in  its  first 
operation.  Reportedly,  the  system  was  not 
radically  new  in  concept,  but  it  used  good 
standard  displays  and  allowed  efficient 
handling  of  increasing  or  decreasing 
traffic  loads.  One  American  pronounced  the 
demonstration  system  the  "best  in  the 
world";  Western  documentation  of  it  is  not 
yet  widely  available.  From  recent  Soviet 
news  reports,  it  appears  that  Russia  will 
attempt  to  duplicate  the  Swedish  system  at 
many  other  sites,  but  with  Russian-built 
hardware. 

Falzon  (Institut  National  de  Recherche 
d' Informat ique  et  d'Automatique,  Rocquen- 


court,  France)  has  an  interesting  propos¬ 
al  for  unburdening  air  traffic  control¬ 
lers.  He  starts  with  a  basic  air-control 
problem  (possible  conflict  between 
predicted  flight  paths  of  two  or  more 
aircraft) ,  and  compares  the  physical 
representation  of  events  with  the  opera¬ 
tor's  mental  representation.  The  two 
forms  of  representation,  it  turns  out, 
are  quite  different.  To  resolve  the 
disparity,  the  basic  radar  data  are 
transformed  into  a  special  "conflict 
zone"  display;  the  operator  only  has  to 
watch  whether  a  "bug"  is  approaching  a 
"conflict  zone"  on  the  display.  Judg¬ 
ment,  the  aiding  process  assumes,  will  be 
easier  and  more  reliable  than  the  typical 
air-controller  mental  operation. 

Determining  the  mental  representa¬ 
tion  is  a  general  and  intriguing  problem 
in  such  schemes.  There  seems  to  be  no 
elegant  or  unique  way  of  solving  it, 
though  several  European  workers  have  some 
ideas.  At  Aston  Univ.  (Birmingham,  UK), 
for  example,  the  mental  picture  that  air 
controllers  use  was  explored  by  watching 
the  kinds  of  information  that  controllers 
convey  to  each  other  when  they  hand  over 
a  sector  to  another  controller  at  shift 
change  time. 

Various  control  studies  at  Wacht- 
berg-Werthoven,  West  Germany  (Forschung- 
institut  fur  Anthropotechnik)  ,  have  been 
aimed  at  unburdening  the  operator  by 
doing  some  routine  work  for  him.  A  good 
illustration  is  Weddel's  study  of  subma¬ 
rine  depth  and  heading  control.  In 
accord  with  American  results,  when  a 
predictor  display  was  added  to  the 
control  configuration,  performance  was 
much  better,  and  the  experiments  showed 
strong  interactions  between  aiding  and 
personnel  selection  variables.  That  is, 
if  you  are  using  a  standard  set  of 
displays,  people  with  high  "space"  scores 
perform  better  than  those  with  low 
scores.  But  when  the  predictor  capabil¬ 
ity  is  added,  the  difference  between 
"high  space"  and  "low  space"  people 
vanishes,  suggesting  that  aiding  can 
compensate  directly  for  individual 
ability  differences.  Predictor  and 
"augmenting"  displays  for  crop-duster 
aircraft  control  activities  have  also 
proved  effective  in  Israeli  simulators 
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(Department  of  Aeronautical  Engineering, 
Technion).  From  this  work,  it  seems  that 
the  operator  does  not  need  extreme  realism; 
rather,  he  needs  a  dynamic  scene  represen¬ 
tation  that  is  structurally  unambiguous, 
that  is  active  enough  to  reflect  the  time 
relations  in  the  real  environment,  and  that 
does  not  impose  a  lot  of  secondary  work 
(such  as  dial  reading)  on  the  human  con¬ 
troller. 

"Unburdening"  of  handicapped  people, 
such  as  the  deaf  or  paralyzed,  is  being 
pursued  in  every  European  country.  Lip- 
reading,  for  instance,  is  extraordinarily 
difficult  even  for  deaf  subjects  who  have 
much  practice;  it  takes  the  deaf  person's 
fullest  attention  to  get  a  meaningful 
fraction  of  an  utterance  from  another 
person.  IBM  France  (Paris)  has  a  long¬ 
term  project  on  computerized  aids  that 
should  help  to  disambiguate  the  lip-move¬ 
ment  signal  (a  "b"  and  "p"  have  identical 
visual  lip  stimuli,  but  can  be  distin¬ 
guished  by  on-line  acoustic  signal  analy¬ 
sis).  If  successful,  the  IBM  scheme 
eventually  should  provide  the  same  kind  of 
supplementary  information  as  "Cued  Speech" 
(manual  supplementary  cuing)  or  the  Danish 
mouth-hand  coding  technique.  IBM's  ap¬ 
proach  is  quite  general  and  should  adapt 
quickly  to  any  normal  speaker.  Perhaps 
best  of  all,  the  normal  speaker  does  not 
have  to  learn  a  complex  hand-movement  code 
such  as  Cued  Speech  in  order  to  communicate 
with  a  deaf  subject.  There  are  also 
European  systems  for  helping  subjects 
control  objects  and  make  keyboard  entries 
by  slight  head  or  mouth  movements.  A 
joystick-controlled  car  steering  arrange¬ 
ment  has  been  tested  in  Malvern,  England, 
and  is  now  being  supplied  to  some  Thalido¬ 
mide  victims;  with  the  device,  a  car  can  be 
driven  with  only  one  good  hand,  one  good 
foot,  or  even  one  finger. 

Unburdening  is  based  on  the  idea  of 
mental  capacity;  the  aim  is  to  unburden  a 
human  operator  when  his  capacity  is  ex¬ 
ceeded.  (You  may  also  want  to  supply  new 
stimuli  to  alert  someone  if  the  information 
load  is  low  enough  to  encourage  boredom  and 
inattentive  behavior.)  There  is  some 
elegant  European  work  on  the  quantification 
of  human  information-processing  capacity. 
At  the  Wachtberg-Werthoven  simulators  in 
Germany,  three  measures — transinformation 


rate,  control  stick  activity,  and  eye 
fixation  on  a  secondary  task — were 
combined  to  yield  an  attractive  index  of 
mental  workload.  In  the  same  laboratory, 
a  difference-equation  model  of  the  ILS 
approach  sequence  was  validated;  it  is 
one  of  the  few  uses  of  difference-equa¬ 
tion  models  in  human  control  performance. 
Dutch  investigators  have  been  able  to 
quantify  uncertainty  in  car  driving 
behavior;  Godthelp,  Milgram,  and  Blaauw 
(TNO  and  National  Aerospace  Laboratory, 
NLR)  have  even  shown  that  driver  subjects 
seek  a  relatively  constant  information 
gain,  up  to  about  120  km/hr  vehicle 
speed.  Milgram  (NLR),  in  one  of  the 
finest-grained  correlations  yet  of 
cognitive  information  with  physiology, 
showed  that  instantaneous  heart-rate 
changes  vary  with  demand  for  "glimpses" 
of  road  information  in  a  driving  task. 

Researchers  at  Munich  (Anthropotech- 
nik  MBB-GmBH)  have  defined  limits  for 
aircraft  handling  quality  with  a  simpli¬ 
fied  flight  regime.  An  aiding  implica¬ 
tion  of  this  work  is  that  if  demand 
characteristics  such  as  damping,  reso¬ 
nance,  and  time  constants  fall  outside 
the  specified  limits,  then  those  features 
would  have  to  be  aided  for  reliable 
performance.  No  doubt  some  of  the  limits 
from  Munich  will  appear  in  future  design 
handbooks. 

In  Adrano,  Sicily,  an  unusual  solar 
power  plant  now  operates  with  a  comput¬ 
er-aided  unburdening  system  and  only  one 
or  two  human  monitors.  The  plant  itself 
is  unique:  a  mirror  field  focuses  the 
sun's  energy  on  a  receiving  tower,  and 
the  heat  energy  is  then  used  to  drive  a 
steam  generator  and  turbine.  Unlike 
conventional  power  plants,  which  operate 
in  a  fairly  steady  condition  most  of  the 
time,  a  solar  power  operation  is  very 
transient.  Clouds  and  other  meteorolo¬ 
gical  conditions  can  cause  immediate 
changes  in  pressure  and  temperature.  In 
fact,  during  the  early  operation  of  the 
plant  the  operators’  control  strategy  was 
simply  to  take  a  walk  and  look  around, 
all  the  while  using  a  mental  model  of  the 
operating  logic.  The  procedure  was 
apparently  quite  stressful  because  the 
operators  had  to  watch  many  variables  and 
had  to  make  frequent  judgments  about  the 
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selection  of  immediate  control  maneuvers. 
The  operators  often  felt  that  their  capa¬ 
city  was  exceeded. 

After  some  analysis,  an  aiding  system 
was  devised  which,  for  example,  separated 
the  collection  and  processing  of  plant 
data.  A  special  troubleshooting  guide  was 
prepared  for  the  operators.  It  was  inter¬ 
esting  that  although  the  system  itself  was 
modeled  for  engineering  purposes,  for  some 
time  nobody  took  the  trouble  to  convert  the 
abstract  model  into  a  form  accessible  to 
humans.  The  conversion  was  not  done  until 
it  shown  to  be  essential. 

West  German  laboratories  have  produced 
several  excellent  schemes  for  unburdening 
operators  in  aircraft  control,  continuous 
process  manufacturing,  and  power  plants. 
Perhaps  the  most  general  test  bed  is  at  the 
Institute  of  Industrial  Engineering  in 
Stuttgart,  where  the  investigators  can 
segregate  and  simulate  functions  such  as 
data  management,  data  collection,  schedul¬ 
ing,  supervision  of  production  units,  and 
tracking  of  particular  jobs.  Unburdening 
aids  can  reduce  the  error  rates  for  each 
task.  The  Karlsruhe  National  Laboratory 
(Kernforschungzentrum)  has  a  multidisci¬ 
plinary  program  with  a  long-term  aim  of 
designing  truly  flexible  manufacturing 
systems.  The  Karlsruhe  research  indicates 
that  systems  must  incorporate  human  factors 
unburdening  along  several  dimensions  so 
that  work  will  be  both  productive  and 
satisfying  to  everybody  in  the  system.  At 
Braunschweig  (Institut  fur  Flugfiihrung)  it 
has  been  established  that  in-flight  unbur¬ 
dening  aids  for  an  aircraft  crew  can  have 
negative  consequences.  For  example, 
software  failures  are  not  easily  detected 
or  dealt  with  in  . flight.  Unlike  some 
aiding  projects,  the  Braunschweig  effort 
has  been  pursuing  an  individual  differences 
approach  to  decision  aids  in  aircraft.  To 
cite  just  one  point,  older  pilots  who  were 
"raised"  on  semimanual  in-flight  planning 
may  not  welcome  and  may  not  trust  newer 
computerized  technologies  as  fully  as  do 
pilots  who  start  out  their  careers  with 
such  aids. 

Several  organizations  are  doing  work 
on  "software  ergonomics,"  generally  stress¬ 
ing  the  design  of  terminals  and  computer 
dialogues  that  are  transparent  to  relative¬ 
ly  naive  users.  Loughborough  Univ.  (UK)  is 


a  leading  laboratory  in  this  area;  it 
maintains  perhaps  the  only  "electronic 
journal"  in  the  human  factors  area  and 
has  published  several  symposia  and 
handbooks  on  man-computer  interaction. 
There  is  also  a  big  man-machine  interface 
project  at  Leicester  Polytechnic.  The 
Medical  Research  Council  Applied  Psychol¬ 
ogy  Unit  (Cambridge,  UK)  has  a  long-term 
contract  with  IBM-UK  on  the  subject  of 
user-friendly  systems;  several  experi¬ 
ments  on  encoding  and  computer  "search¬ 
ing"  have  been  reported.  For  financial 
accounting  systems,  a  leading  resource  is 
the  Lehrstuhl  fur  Betriebs  Informatik 
(Dortmund,  West  Germany).  Methods  have 
been  applied  there  for  scoring  the 
quality  of  presentation  and  the  adequacy 
of  a  system  for  handling  human  errors  and 
dialogue.  At  the  Research  Group  on 
Man-Machine  Communication  (Stuttgart, 
West  Germany)  Illich's  attractive  idea  of 
a  convivial  system  is  under  serious 
exploration.  In  a  convivial  system,  the 
old  distinction  between  "programming"  and 
"using  a  program"  becomes  blurred.  The 
system  changes  to  accommodate  the  needs 
of  the  user,  who  perceives  himself  as  the 
controlling  agent  and  not  just  as  the 
passive  manipulator  of  a  complex  tool. 

To  summarize  the  status  of  European 
"unburdening"  aids,  one  might  say  that 
for  large  C2  and  C3  systems,  Europe  is 
appreciably  behind  the  US — except  possib¬ 
ly  for  a  few  display  ideas.  In  other 
applications ,  the  models  and  medium-size 
systems  are  often  comparable  in  quality 
to  the  best  work  in  the  US,  and  are 
surely  worth  consideration  by  American 
specialists.  In  a  few  theoretical  areas, 
such  as  human  information  processing 
capacity,  European  work  is  at  the  edge  of 
the  art. 

Diagnostic  Systems 

There  are  diagnostic  systems  every¬ 
where  in  Europe,  for  the  same  reason  they 
are  found  everywhere  in  America:  com¬ 
plexity.  Nobody  can  possibly  remember 
or,  even  know,  all  the  physical  relations 
in  a  modern  guided  missile,  radar,  or 
refinery.  Some  separation  into  manage¬ 
able  chunks  must  be  done.  This  can  take 
place  gradually  in  conventional  power 
plants  and  ship  engine  rooms,  where  the 
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separation  has  evolved  over  the  years  into 
an  arrangement  that  is  reasonably  workable. 
Ke  mention  here  a  few  diagnostic  systems 
that  go  at  least  a  little  beyond  ordinary 
test  sequencing. 

The  "expert  system"  idea  is  current  in 
Europe.  The  British  Computer  Society  has  a 
special  panel  on  expert  systems.  It  is 
still  true,  though,  that  few  expert  systems 
are  actually  on  line.  One  of  the  most 
promising  is  the  Pourbaix-Michie  project 
(Oxford,  UK)  on  metal  corrosion.  Pourbaix, 
a  Belgian  metallurgical  researcher  of 
international  standing,  has  tried  to  put 
together  what  he  knows  about  metal  corro¬ 
sion;  the  knowledge  base  is  a  combination 
of  many  emperical  curves,  along  with  rules 
for  going  from  one  curve  or  region  to 
another.  The  expert  system  idea,  of 
course,  is  to  assemble  all  this  information 
so  that  it  is  accessible  to  an  ordinary, 
trained  person,  and  so  that  the  outcomes  of 
the  system  resemble  closely  those  Pourbaix 
would  deliver  himself,  were  he  faced  with  a 
complex  prediction  problem.  An  expert 
system  may  also  be  arrayed  so  as  to  "ex¬ 
plain  itself"  in  response  to  queries  about 
the  basis  for  its  inference;  thus  a  medical 
aid  might  cite  journal  references  for  the 
treatment  recommendations  it  makes  in  a 
particular  inference  chain.  Among  many 
other  attempts  to  encapsulate  expert 
knowledge  are  the  Cancer  Prediction  project 
(IBM,  Paris,  France)  and  the  anesthesiology 
system  at  Aachen,  Germany  (Helraholtz-Insti- 
tut  fur  Biomedizoniche  Technik).  The 
latter  laboratory  has  done  a  rather  com¬ 
plete  analysis  of  the  anesthesiologist's 
task;  the  present  version  is  designed  for 
two  humans  (process  controller  and  process 
state  operator);  it  can  offer  advice  on 
state  hypotheses  and  possible  actions,  say 
in  the  case  of  an  unexpected  drop  in  blood 
pressure.  Anesthesiologists  usually  know 
what  to  do  in  such  cases,  but  if  some 
parameters  are  sampled  and  their  relations 
satisfactorily  modeled  and  instantaneously 
"solved,"  it  should  be  possible  to  make 
decisions  and  take  action  more  efficiently 
in  special  conditions. 

Many  diagnostic  aids  in  European  power 
plants  have  been  influenced  by  the  Rasmus¬ 
sen  (RISO,  Denmark)  three-level  model  of 
troubleshooting  behavior.  The  model 
distinguishes  knowledge-based,  rule-based, 


and  skill-based  behaviors.  It  is  rather 
convenient  for  computerized  aiding,  as 
the  three  types  of  actions  are  more  or 
less  discernible  in  the  task  organization 
of  many  power  stations.  German  analyti¬ 
cal  work  in  the  area  is  often  quite 
elegant.  At  Garching,  West  Germany 
(Gesellschaft  ftir  Reakforscherheit) , 
investigators  have  used  formal  language 
structures  to  bridge  the  human  factors 
and  hardware  domains.  When  applied  to  a 
suitable  data  base,  a  node  structure 
calls  up  the  possible  causes  associated 
with  any  disturbances  in  the  plant.  It 
might  be  especially  useful  when  many 
alarms  are  set  off  at  once;  the  aid 
could,  for  example,  indicate  which  alarms 
the  human  should  be  monitoring,  and  which 
can  be  ignored.  Some  of  the  same  ideas, 
though  the  realizations  are  different, 
can  be  observed  in  the  Ispra  work  on 
nuclear  reactor  operators  (CEC  Joint 
Research  Centre,  Ispra,  Italy). 

Fuzzy  set  diagnostic  models  have 
often  been  proposed  in  Europe,  and  a  few 
of  them  may  be  working  practically  soon. 
The  International  Journal  of  Man-Machine 
Studies  is  a  major  source  of  theory  and 
projects  in  this  domain,  with  Gaines 
(Manchester  Univ.)  as  the  leading  investi¬ 
gator  in  Europe. 

French  mathematicians  have  inspired 
several  diagnostic  schemes  based  on 
concepts  from  information  theory.  One 
example  is  a  French  program  that  pre¬ 
scribes  "next  best"  tests  for  kidney 
patients,  given  the  results  of  all 
previous  tests.  Again,  the  theory  seems 
to  be  ahead  of  applications,  as  it  is 
difficult  to  find  a  hospital  that  uses 
such  programs,  despite  their  theoretical 
efficiency. 

Danish  investigators  (Technical 
Univ.,  Lyngby)  have  an  aid  for  those  who 
must  design  and  run  logic  simulation 
programs  on  new  digital  systems.  If 
there  are  hundreds  of  gates  in  a  circuit, 
breadboarding  is  impractical,  and  all  the 
node  labeling  must  be  done  manually — with 
a  substantial  likelihood  of  errors.  The 
Lyngby  aiding  program  accepts  some  16 
standard  gate  symbols,  keeps  track  of 
everything,  and  is  handy  for  checking  the 
circuit  outputs  and  tracing  faults.  For 
the  labeling  task  alone,  man-hour  savings 
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on  the  order  of  75  to  80%  have  been  re¬ 
ported,  ind  the  incidence  of  labeling 
errors  has  been  sharply  reduced.  When  the 
entire  circuit  is  simulated,  printed 
records  of  all  the  outputs  provide  good 
documentation  for  designers  who  will  be 
using  the  circuit  in  a  real  device. 

An  overview  of  European  diagnostic 
aids  suggests  that  a  portable  configura¬ 
tion,  or  perhaps  a  stand-alone,  desk-size 
hardware  setup,  soon  will  be  the  standard 
arrangement  for  practical  aiding.  Already 
in  domains  like  plant  disease  analysis  and 
specialized  medical  problems,  the  analyst 
often  goes  right  to  the  scene  with  a 
portable  computer  and  a  handful  of  disks 
and  manuals.  Some  aids  still  talk  to  a 
central  computer  over  a  telephone-linked 
modem,  but  as  microcomputer  technology 
advances,  nearly  all  aiding  systems  should 
be  portable.  Customers  and  field  users 
like  portability  because  they  feel  as 
though  they  have  their  own  system. 

There  seem  to  be  few  really  new 
methods  for  diagnosis.  Techniques  like 
fault  trees,  symptom-fault  matrices, 
branched  check  sequences,  and  probabil- 
ity-of-failure  tables  continue  to  be  the 
backbone  of  diagnostic  aids.  One  major 
trend  is  noticeable;  nearly  all  major 
researches  into  diagnostic  aids  are  moving 
closer  to  more  human-oriented  models  of 
complex  systems  such  as  electronic  devices 
and  process  plants.  The  old  way  was  to 
represent  the  basic  plant  with  diagrams 
that  were  suitable  for  engineering  and 
design  purposes;  technicians  had  to  learn 
enough  to  use  these  rather  abstract  mater¬ 
ials.  Certain  displays  and  indicators  were 
inserted  into  the  plant  design  at  strategic 
places,  and  the  operators  gradually  learned 
how  to  control  and  troubleshoot  the  plant 
by  watching  test  instruments  or  big  panels 
of  dials  and  gauges. 

A  newer  and  probably  better  way  is  to 
discover  how  a  person  can  naturally  carry 
"the  picture"  of  the  plant  as  a  series  of 
mental  images,  and  then  to  provide  displays 
and  procedures  that  are  compatible  with  the 
remembered  mental  model.  In  fact,  American 
researchers  might  find  that  the  efforts  to 
explore  the  mental  models  actually  formula¬ 
ted  and  used  by  the  operators  and  techni¬ 
cians  are  among  the  Europeans’  most  valu¬ 
able  contributions  to  diagnostic  aics. 


Structuring  Systems 

For  our  purposes,  a  "structuring" 
aid  does  not  necessarily  compute  a  fault 
tree  or  tell  you  what  to  do  next;  rather, 
if  helps  you  to  examine  intensively  your 
own  (and  perhaps  evolving)  trade-offs  and 
reactions  to  various  alternatives  in  a 
decision  problem.  The  Harvard  Business 
School  has  perhaps  the  most  famous  aiding 
"package"  of  this  sort.  Many  teachers 
and  practitioners  have  used  it  to  imple¬ 
ment  the  Schlaif er-Raif fa  decision  model. 
Using  gambles  and  certainty  equivalents, 
one  can  estimate  utility  functions.  To 
exemplify  European  work  in  this  area,  we 
describe  briefly  the  MultiAttribute 
Utility  Decomposition  (MAUD)  program, 
which  has  been  produced  and  applied  by 
Phillips  and  Humphreys  at  the  London 
School  of  Economics. 

MAUD  is  a  truly  interactive  scheme; 
it  works  on  a  small  computer  (some  new 
portable  microcomputers  could  accommodate 
it).  To  get  things  started,  it  requires 
the  user  to  name  the  decision  alterna¬ 
tives  and  to  provide  the  attributes  or 
features  relevant  to  the  alternatives. 
"Poles"  on  each  end  of  an  attribute 
dimension  are  also  defined  in  verbal 
terms  acceptable  to  the  decision-maker. 
Then  MAUD  has  the  decision-maker  rate  all 
the  alternatives  on  the  scale  enclosed  by 
the  poles  and  requires  the  identification 
of  "ideal  points"  on  each  dimension. 
MAUD  automatically  converts  the  prelimin¬ 
ary  numbers  into  0  and  1  values  for  least 
and  most  preferred  alternatives. 

MAUD  goes  to  great  trouble  to 
encourage  "conditional  utility  indepen¬ 
dence"  among  the  rated  dimensional 
values.  (Xj  is  conditionally  utility 
independent  of  X2,  given  X3...Xn,  if 

Xj  is  utility  independent  of  X2  for 
all  fixed  levels  of  X3...X  .)  To 

assist  in  this  search  for  additive 
independence,  the  program  watches  the 
statistical  correlations  among  ra.ings, 
displays  to  the  decision-maker  variables 
exhibiting  appreciable  correlate  ,  and 
may  suggest  redefinition  or  del.  on  < f 
dimensions.  After  all  the  a'  “r.nls 
have  been  made  and  both  MAUD  ana  the  user 
seem  fairly  well  satisfied  w-'h  the 
structure,  MAUD  goes  on  formu- 
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late  explicit  trade-off  comparisons.  The 
program  uses  Raiffa-BRLT  gambles  to  quan¬ 
tify  the  trade-off  judgments.  The  final 
output  from  MAUD  is  an  estimated  utility 
for  each  alternative,  along  with  a  printout 
of  all  the  dimensions  and  ratings.  After 
seeing  the  summary  material,  the  decision¬ 
maker  may  elect  to  change  some  of  the 
structure  and  values.  If  everything  has 
worked  rather  well,  MAUD's  output  can  serve 
as  a  very  good  breakdown  of  a  complex 
decision  problem.  Certainly  MAUD  includes 
many  techniques  that  decision  analysts  have 
explored  over  the  past  two  decades. 

Other  structuring  packages  and  models 
have  been  put  together  in  Europe.  Among 
the  leading  centers  is  the  Helsinki  School 
of  Economics  and  Business  Administration 
(JyvMkyia,  Finland);  investigators  there 
have  an  ingenious  solution  to  the  problem 
of  the  quality  of  a  solution  that  is 
derived  from  only  a  subset  of  all  possible 
comparisons.  The  Technical  Univ.  at 
Leuna-Merseburg  (East  Germany)  has  a 
package  called  DIDASS,  which  takes  the  user 
through  several  stages  (define  the  problem 
structure,  modify  the  structure,  display 
the  weights).  But  instead  of  interactive 
BRLT  ratings,  the  decision-maker  keeps 
working  on  a  goal  point  or  reference  point 
reflecting  the  human's  "aspiration  levels" 
on  the  different  dimensions.  Because 
multicriteria  aspirations  are  often  contra¬ 
dictory  and  cannot  be  achieved  simulta¬ 
neously,  DIDASS  focuses  on  the  jet  of 
not-inferior  points,  and  allows  the  opera¬ 
tor  to  shift  things  about  until  a  point 
nearest  the  desired  goal  is  attained. 
DIDASS  software  is  not  yet  easily  obtained 
in  the  West.  Some  European  groups  (e.g.. 
Decision  Theory  Department,  Univ.  of 
Manchester)  have  been  critiquing  the  basic 
feasibility  and  validity  questions  in  any 
structuring  system.  The  conclusion  is  that 
many  of  the  assumptions  underlying  all  the 
scaling  are  shaky  indeed.  A  few  packages 
can  accept  and  process  fuzzy  linguistic 
variables  and  can  calculate  the  utility  of 
intermediate  values  by  interpolation. 
There  also  are  many  small  manual  aids.  An 
example  is  Hawgood ' s  ABACON  chart  (Univ.  of 
Durham,  UK);  though  seldom  cited  by  Ameri¬ 
can  researchers,  it  can  be  used  to  cali¬ 
brate  a  utility,  and  it  could  be  compu¬ 
terized  . 


Interactive  Graphics  in  Aiding  Systems 

It  is  often  hard  to  know  where  a 
given  discovery  in  graphics  technology 
originated;  this  is  because  American, 
Japanese,  and  European  graphics  packages 
are  now  on  sale  all  over  the  world. 
Several  such  packages  were  demonstrated 
at  a  pattern  recognition  conference  in 
Munich  during  October  1982,  and  at  the 
Berlin  Computer  Graphics  meeting  in  March 
1983.  With  the  Japanese  "Spider"  pack¬ 
age,  for  example,  an  analyst  could 
immediately  compare  the  effectiveness  of 
the  standard  pictorial  transformations 
(Preuitt,  Hueckel,  etc.)  as  they  proces¬ 
sed  a  complex  scene.  One  well-known 
analytical  method,  the  Bezier  curve 
system,  is  strictly  a  European  develop¬ 
ment.  It  was  originated  by  P.  Bezier  of 
the  Renault  automotive  firm  and  is 
realized  commercially  in  the  UNISURF 
package.  The  program  uses  a  "blending 
function"  to  weight  the  effect  that  each 
control  point  exerts  on  a  curve  that 
passes  through  (or  near)  a  given  set  of 
points.  It  can  quickly  draw  "Bezier 
surfaces"  from  limited  information.  Line 
drawings  of  car  bodies  and  ship  contours 
produced  in  this  way  are  quite  striking, 
and  often  beautiful. 

Some  spline-surface  packages  can 
perform  similar  graphic  functions  but  are 
not  specifically  European.  Cinematic 
animation  laboratories  in  England  and 
France  are  producing  impressive  commer¬ 
cial  work,  but  much  is  still  done  by 
hand,  and  the  exact  processes  are  .propri¬ 
etary.  American  animation  is  probably 
more  computerized.  Darmstadt's  Tech- 
nische  Hochschule  has  a  "Graphic  Kernel" 
system  that  reportedly  permits  the  easy 
modification  of  object  attributes. 
Karlsruhe's  Fraunhofer  Institute  is 
another  important  center  for  interactive 
image  processing.  Researchers  there  have 
many  contrast  and  contour  enhancement 
schemes,  and  some  of  them  have  been 
tested  on  real  aerial  images. 

The  geometry  for  displaying  complex 
three-dimensional  scenes  on  a  two-dimen¬ 
sional  display  is  well  known,  and  one  can 
compute  quite  adequate  three-dimensional 
and  perspective  transformations  of 
well-defined  objects.  (Recent  ONR-spon- 
sored  research  indicates  that  human 
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perception  of  objects  on  such  displays  can 
be  subject  to  various  context  and  compensa¬ 
tion  effects.)  Of  course,  a  major  problem 
is  the  computational  power  required  to 
produce  and  to  update  the  pictorial  dis¬ 
plays.  Millions  of  multiplications  per 
minute  or  second  are  necessary  to  provide  a 
dynamic  digital  simulation,  in  real  time, 
of  a  specific  airport  area.  Fortunately, 
as  trial  and  experimentation  proceed, 
tricks  for  reducing  the  computational 
requirements  are  being  discovered  and 
rediscovered,  and  display  designers  are 
learning  more  about  what  is  really  needed 
for  an  effective  presentation.  At  the  same 
time,  computing  and  storage  facilities 
continue  to  increase.  Perhaps  the  most 
general  scheme  for  3-D  analysis  of  moving 
objects  in  the  Nagel  project  at  the  Univ. 
of  Hamburg.  Starting  from  a  bare  minimum 
of  assumptions  (e.g.,  rigidity  of  the 
object),  Nagel's  system  discriminates  the 
moving  objects  in  a  complex  scene  from  the 
background;  it  even  manages  to  compute 
visual  edges  and  shaded  planes  of  the 
convex  hull  of  the  3-D  point  configuration. 

It  is  easy  to  foresee  quite  general 
graphics  packages  that  will  run  on  medium¬ 
sized  aiding  systems;  some  are  available 
now,  and  within  a  year  or  two  there  will  be 
many  choices.  The  systems  will  permit  many 
physical  events  and  decision  structures  now 
displayed  in  verbal  or  numerical  form  to  be 
portrayed  in  a  vivid  graphic  format  and  to 
be  updated  in  something  like  real  time. 
Tactical  displays,  for  instance,  could  be 
shown  in  a  more  "solid"  form,  perhaps  in, 
say,  an  animated  cartoon  format.  No  doubt 
such  displays  will  be  well  received  by  many 
users;  sometimes  there  might  even  be  "too 
much"  graphics.  The  scientific  community 
needs  hard  experimental  data  on  the  utility 
of  such  presentational  technology.  If 
diagnostic  trees,  or  the  intermediate 
working  operations  of  structuring  programs, 
are  shown  in  dynamic  three-dimensional 
lattices,  will  the  human  judgments  be 
improved?  And  what  about  the  effects  of 
graphics  upon  group  judgments  and  group 
decision  making?  Fortunately,  the  means  to 
answer  such  questions  seem  available,  and 
it  is  to  be  hoped  that  the  answers  will  be 
pursued  as  vigorously  as  is  the  engineering 
in  the  new  displays. 


The  Criterion  Problem 

Logically,  a  decision  aid  could  be 
evaluated  by  comparing  human  decisions 
when  the  aid  is  used  with  those  made  in 
the  unaided  situation.  Different  aiding 
systems  could  also  be  rated  for  qualities 
such  as  user  acceptability,  transparency, 
and  cost;  in  1982  a  NATO  conference  in 
Mons,  Belgium,  devoted  much  attention  to 
these  matters.  However,  questions  of 
decision  quality  arise  immediately,  and 
in  some  military  settings  these  issues 
can  be  approached  only  through  gaming  and 
simulation.  One  European  defense  minis¬ 
try  has  adopted  a  multi-attribute  utility 
approach  to  evaluating  tactical  aids  at 
the  battalion-and-up  level.  An  early 
result  is  that  when  the  information  level 
is  above  a  certain  threshold,  then  more 
information  and  aiding  are  of  little  use, 
but  until  then,  an  aiding  system  is  quite 
worthwhile.  This  intriguing  result 
deserves  further  validation.  It  seems  to 
consist  of  more  than  the  simpleminded 
observation  that  once  the  variances  in 
controlling  parameters  are  fairly  small, 
then  any  systematic  procedure  for  combin¬ 
ing  data  is  good  enough. 

One  problem  has  been  recognized 
throughout  Europe  but  has  not  been  well 
researched:  certain  high-status  deci¬ 
sion-makers — e.g.,  doctors,  military  and 
political  leaders,  and  commercial  execu¬ 
tives — resist  aiding  in  their  jobs. 
Apparently  many  top  decision-makers 
prefer  to  do  the  final  "assembling"  of 
problem  parameters  themselves.  One 
example  is  that  military  leaders  often 
want  their  systems  and  aids  to  provide 
probabilities  of  certain  events  and 
states,  but  do  not  want  these  to  be 
multiplied  with  utilities  and  presented 
in  fully  processed  "recommendation"  form. 
If  such  behavior  reflects  the  decision¬ 
maker's  suspicion  of  the  "output  numbers" 
in  an  aiding  system,  the  skepticism  might 
be  reduced  if  the  system  were  more 
transparent.  But  the  decision-makers 
simply  might  want  to  make  their  own 
choices,  and  thereby  exert  control  and 
demonstrate  responsibility.  The  rioblems 
are  difficult,  but  they  will  hr  to  be 
faced  by  aiding  system  designers — or  at 
least  by  those  who  are  genuinely  con¬ 
cerned  with  the  use  of  the  pro  in  ts. 
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Though  it  may  not  be  well  publicized, 
the  experience  with  commercial  decision- 
support  packages  should  be  applicable  to 
both  military  and  technological  areas.  As 
far  as  this  writer  knows,  no  European 
company  or  computer  service  firm  has 
assembled  a  representative  set  of  cases 
regarding  the  commercial  impact  of  deci¬ 
sion-support  systems.  What  we  often  get  is 
a  collection  of  anecdotes  regarding  certain 
notable  cases.  Perhaps  the  state  of  the 
art  is  such  that  both  users  and  producers 
of  the  systems  are  still  coming  to  terms 
with  them — a  process  that  may  take  a  few 
more  years. 

The  implementation  of  decision  aids  is 
another  area  in  which  there  are  many 
anecdotes  but  little  solid  analysis. 
Scientists  and  technologists  who  produce 
aids  often  put  their  best  efforts  into 
items  that  "obviously"  can  assist.  It  is 
then  assumed  that  use  will  follow  because 
the  device  is  clearly  helpful,  and  people 
need  the  help.  But  we  know  that  many 
effective  aids  simply  are  not  used. 

On  the  positive  side,  in  some  situa¬ 
tions  the  use  of  aids  is  the  most  important 
part  of  the  job — in  the  troubleshooting  of 
big  commercial  computers,  for  example.  A 
great  majority  of  main-frame  troubles  are 
corrected  quickly  by  running  special 
diagnostic  programs  and  replacing  one  or  a 
few  component  cards  or  units.  The  arrays 
of  logic  gates  are  tailor-made  for  aided 
diagnosis;  in  fact,  a  defective  card  often 
can  be  located  in  a  few  minutes.  Signifi¬ 
cantly,  some  of  the  worst  troubles  with 
computer  systems  now  involve  electro-me¬ 
chanical  equipment,  which  is  intrinsically 
less  suitable  to  techniques  such  as  logic 
tables  and  tree  searches.  For  big  com¬ 
puters,  the  conditions  favoring  implementa¬ 
tion  of  an  aid  are  rather  few  and  obvious: 


first,  you  design  your  aids  so  that  they 
really  do  work.  Then  you  give  special 
training  in  their  use,  maintain  super¬ 
visory  pressure  to  use  and  to  update 
them,  keep  a  technically  competent,  but 
not  elite,  corps  of  technicians  to  apply 
them,  and  prepare  troubleshooting  aids  as 
an  integral  part  of  the  hardware  design. 
(If  possible  faults  cannot  be  found  by 
the  aid,  the  hardware  is  redesigned  until 
they  can  be.)  In  policy  analysis,  there 
is  some  American  and  European  theoretical 
work  on  the  sociology  of  implementation. 
For  example,  a  constituency  analysis  of 
the  potential  users  and  maintainers  of 
programs  can  illuminate  the  factors 
underlying  use  of  a  new  technology.  It 
would  appear  worthwhile  for  a  thorough 
project  on  the  implementation  of  decision 
aids  to  be  carried  out,  perhaps  Liv  ONR 
442EP  contractors  who  have  already 
produced  submarine,  aircraft,  and  elec¬ 
tronic  warfare  aids  in  the  early  1980s. 
For  example,  how  many  aiding  items 
produced  under  ONR  contract  are  now  in 
continual  use? 


Conclusion 

This  review  has  discussed  many 
interesting  decision-aiding  projects  in 
Europe,  but  few  really  innovative  ones. 
As  one  of  the  world's  leading  agencies 
for  sponsoring  research  in  this  field, 
ONR  is  in  a  good  position  to  stay  abreast 

of  new  developments. _ Specific  ONR 

programs  on  mental  models  of  complex 

processes , _ knowledge-representation 

schemes,  and  interactive  graphics  pro¬ 
bably  will  have  both  scientific  and 
military  payoff;  these  programs  will 
benefit  from  continual  exchanges  of 
technical  information  between  American 
and  European  projects. 
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